Application of machine learning to identify risk factors for 14-day emergency department return visits in pediatric sickle cell patients discharged after vaso-occlusive crises Free

Sickle cell disease (SCD) is a severe, chronic disease associated with frequent vaso-occlusive crises (VOC), leading to extreme pain as red blood cells block local blood flow and oxygen delivery. VOCs are the leading cause of Emergency Department (ED) visits and hospitalizations for SCD children . Nearly a third of children discharged from the ED following a visit for VOC return to the ED within 14 days and more than half require hospitalization.

Our study objective was to apply machine learning to readily available clinical data to predict 14-day return visits for SCD children who are discharged from the ED after treatment for acute VOC. Machine learning is well-suited for prediction problems related to SCD, given its known multidimensional pathophysiology and the need for frequent healthcare visits, providing large amounts of baseline clinical and laboratory data from which to measure deviations from normal. We used single-site retrospective data from a large, urban, pediatric center to identify ED visits for SCD children. We included visits of SCD children who received at least one dose of intravenous opioids and were discharged from the ED. Visits for children with more than 12 visits in the prior year were excluded. We queried data and generated values for predictor variables by combining information from separate SCD and ED clinical registries, with data collected by chart review and automated daily updates from the electronic health record, respectively. Variables with > 5% missing were excluded. To address interactions and collinearity, study investigators reviewed collinear variables and voted on which variable to keep during model development, based on clinical assessment of importance. We used a random 80/20 split to assign visits to development and hold-out validation sets and applied four machine learning algorithms. We used SHapley Additive exPlanations (SHAP) to rank feature importance across all models. Anticipating an imbalanced dataset, we used weighted average F1 score to compare model performance on the validation set, reporting the point estimate for each model.

1489 visits were included in the training set (251 with return visits, 1238 without), the test set included 312 visits where there was no return visit and 61 with a return visit. A total of 77 predictor variables were collected, including demographics (age, sex), and visit-level vital signs (e.g. heart rate and respiratory rate at start and end of visit), laboratory data (e.g. hemoglobin and reticulocyte count), and medication (e.g. weight-adjusted opioid doses). Additional variables were created based on prior-visit data, including number of prior visits and differences between visit-level vital sign and laboratory data and average values over the prior year. Prior to analysis, 32 were removed for high levels of missingness, including variables traditionally considered to be clinically important (e.g. genotype and history of comorbid conditions associated with SCD severity: asthma, history of avascular necrosis and chronic transfusion requirements). A total of 16 predictor variables were included in the models and t, across all models, included number of visits in the prior year, number of visits specifically for VOC in the prior year, and the difference between hemoglobin at the visit and the mean hemoglobin for that patient over the prior year. Considering all visits for children with SCD during the study period, 50% (8600/17,189) were related to pain; aModel performance was highest for simple neural networks (weighted F1 0.77) compared to other models (weighted F1 range: 0.58-0.73).

Machine learning can be applied to structured ED visit data, including data across multiple visits in a given year, to predict return visits among children deemed ready for discharge. Although model discrimination performance is modest, this approach shows promise for possible interventions which might have minimal risk, such as targeting children at high-risk with additional resources to decrease the need for return visits. Additionally, models consistently identify the same top 3 variables, adding to model explainability and emphasizing the importance of using longitudinal data to predict this outcome. Future work should focus on taking advantage of multisite data, using natural language processing to extract information from unstructured clinical notes, and exploring the use of imaging and other multimodal data to improve model predictive performance.